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Preface 



Where is system architecture heading? The special interest group on Computer 
and Systems Architecture (Fachausschuss Rechner- und Systemarchitektur) of 
the German computer and information technology associations GI and ITG ask- 
ed this question and discussed it during two Future Workshops in 2002. The 
result in a nutshell: Everything will change but everything else will remain. 

Future systems technologies will build on a mature basis of silicon and IC 
technology, on well-understood programming languages and software engineering 
techniques, and on well-established operating systems and middleware concepts. 
Newer and still exotic but exciting technologies like quantum computing and 
DNA processing are to be watched closely but they will not be mainstream in 
the next decade. Although there will be considerable progress in these basic 
technologies, is there any major trend which unifies these diverse developments? 

There is a common denominator - according to the result of the two Fu- 
ture Workshops - which marks a new quality. The challenge for future systems 
technologies lies in the mastering of complexity. Rigid and inflexible systems, 
built under a strict top-down regime, have reached the limits of manageable 
complexity, as has become obvious by the recent failure of several large-scale 
projects. Nature is the most complex system we know, and she has solved the 
problem somehow. We just haven’t understood exactly how nature does it. But 
it is clear that systems designed by nature, like an anthill or a beehive or a 
swarm of birds or a city, are different from today’s technical systems that have 
been designed by engineers and computer scientists. Natural systems are flexible, 
adaptive, and robust. They are in permanent exchange with their environment, 
respond to changes adequately, and are very successful in staying alive. It seems 
that also the traditional basic technologies have realized this trend. Hardware is 
becoming reconfigurable, software now updates itself to fulfill new requirements 
or replace buggy components, and small portable systems form ad hoc com- 
munities. Technical systems of this kind are called Organic Computer systems. 
The key challenge here will be to understand and harness self-organization and 
emergence. Organic Computing investigates the design and implementation of 
self-managing systems that are self-configuring, self-optimizing, self-healing, self- 
protecting, context aware, and anticipatory. 

ARCS 2004 continued the biennial series of German Conferences on Archi- 
tecture of Computing Systems. This seventeenth conference in the series served 
as a forum to present current work on all aspects of computer and systems ar- 
chitecture. The program committee of ARCS 2004 decided to devote this year’s 
conference to the trends in organic and pervasive computing. 

ARCS 2004 emphasized the design, realization, and analysis of the emerging 
organic and pervasive systems and their scientific, engineering, and commercial 
applications. The conference focused on system aspects of organic and pervasive 
computing in software and hardware. In particular, the system integration and 
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self-management of hardware, software, and networking aspects of up-to-now 
unconnected devices is a challenging research topic. Besides its main focus, the 
conference was open to more general and interdisciplinary themes in operating 
systems, networking, and computer architecture. 

The program reflected the main topics of the conference. The invited talk of 
Andreas Maier (IBM) presented the Autonomic Computing Initiative sparked by 
IBM which has objectives similar to but not identical with Organic Computing. 
Erik Norden’s (Infineon) presentation discussed multithreading techniques in 
modern microprocessors. 

The program committee selected 22 out of 50 submitted papers. We were 
especially pleased by the wide range of countries represented at the conference. 
The submitted paper sessions covered the areas Organic Computing, peer-to- 
peer computing, reconfigurable hardware, hardware, wireless architectures and 
networking, and applications. 

The conference would not have been possible without the support of a large 
number of people involved in the local conference organization in Augsburg, and 
the program preparation in Hannover. We want to extend our special thanks to 
the local organization at the University of Augsburg, Faruk Bagci, Jan Petzold, 
Mattias Pfeffer, Wolfgang Trumler, Sascha Uhrig, Brigitte Waimer-Eichenauer, 
and Petra Zettl, and in particular to Fabian Rochner of the University of Han- 
nover, who managed and coordinated the work of the program committee with 
admirable endurance and great patience. 
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Keynote 

Autonomic Computing Initiative 



Andreas Maier 

IBM Lab Boblingen 
maieraOde . ibm . com 



Abstract. Autonomic computing systems have the ability to manage 
themselves and dynamically adapt to change in accordance with busi- 
ness policies and objectives. Self-managing environments can perform 
such activities based on situations they observe or sense in the IT en- 
vironment, rather than requiring IT professionals to initiate the tasks. 
Autonomic computing is important today because the cost of technology 
continues to decrease yet overall IT costs do not. With the expense chal- 
lenges that many companies face, IT managers are looking for ways to 
improve the return on investment of IT by reducing total cost of owner- 
ship, improving quality of service, accelerating time to value and manag- 
ing IT complexity. The presentation will outline where IBM comes from 
with its autonomic computing initiative and what has been achieved to 
date. 
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Multithreading for Low-Cost, Low-Power 
Applications 



Erik Norden 



Senior Architect 

Infineon Technologies, Balanstrasse 73, 81541 Munich, Germany 

erik . norden@inf ineon . com 



Abstract. Innovative architectural design may be the best route to cre- 
ate an economical and efficient balance between memory and logic ele- 
ments in cost and power sensitive embedded solutions. When system 
prices are measured in a few euros instead of a few hundred, the large, 
power intensive and costly memory hierarchy solutions typically used 
in computer and communications applications are impractical. A multi- 
threading extension to the microprocessor core is especially for deeply 
embedded systems a more effective approach. 

Infineon has developed a new processor solution: TriCore 2. It is the sec- 
ond generation of the TriCore Unified Processor architecture. TriCore 2 
contains, among others, a block multithreading solution, which responds 
to the blocking code memory latency in one thread by executing the 
instructions of a second thread. In this way, the execution pipelines of 
the processor can be almost fully utilized. From the user programming 
model, each thread can be seen as one virtual processor. 

A typical scenario is a cell phone. Here, generally external 16-bit flash 
memories with a speed of 40 MHz are used while today’s performance 
requirements expect processors with a clock speed of 300-400 MHz. Be- 
cause of this discrepancy, up to 80% of the performance can be lost, 
despite caches. Larger cache sizes and multi-level memory solutions are 
not applicable for cost reasons. 

Block multithreading allows system designers to use comparatively 
smaller instruction caches and slow external memory while still getting 
the same overall performance. The performance degradation in the cell 
phone example can be almost eliminated. Even the clock frequency can 
be reduced. Block multithreading is very efficient for a general CPU 
based application residing in cache memory and an algorithmic applica- 
tion in the local on-chip memory. This is a characteristic which many 
deeply embedded processor applications have. Effectively, a separate 
DSP and CPU can be replaced by a multithreaded hybrid to reduce chip 
area, tool costs etc. The block multithreading solution also supports a 
fast interrupt response, required for most deeply embedded applications. 
The additional costs for this multithreading solution are small. The im- 
plementation in TriCore 2 requires a chip area of only 0.3mm2 in 0.13 
micron technology. The most obvious costs are caused by the duplicated 
register files to eliminate the penalty for task switching. Instruction cache 
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and fetch unit need to support multithreading but the overhead is low. 
Same to other areas that are affected which are traps and interrupt han- 
dling, virtual memory, and debug/trace. 

Apart from multithreading, TriCore 2 has also other highlights. Ad- 
vanced pipeline technology allows high instruction per cycle (IPC) per- 
formance while reaching higher frequencies (400-600 MHz typical) and 
complying with demanding automotive requirements. The center of the 
processor’s hierarchical memory subsystem is an open, scalable crossbar 
architecture, which provides a method for efficient parallel communica- 
tion to code and data memory including multiprocessor capability. 

This presentation will describe the root problems in low-cost, low-power 
embedded systems that require a multithreaded processor solution. 
Since the core architecture is well-deployed in the demanding automotive 
market, the first implementation is specihed for these requirements like 
quality and determinism. Working silicon with this implementation is 
expected for the first quarter of 2004 and will be used as a demonstrator. 



Session I 

Organic Computing 




The SDVM: A Self Distributing Virtual Machine 
for Computer Clusters 



Jan Haase, Frank Eschmann, Bernd Klauer, and Klaus Waldschmidt 



J.W.Goethe-University, Technical Computer Sc. Dep., 

Box 11 19 32, D-60054 Frankfurt, Germany* 

{haase| eschmann| klauer |waldsch}@ti. informatik.uni-frankfurt.de 



Abstract. Computer systems of the future will consist more and more 
of autonomous and cooperative system parts and behave self-organizing. 
Self-organizing is mainly characterized by adaptive and context-oriented 
behaviour. 

The Self Distributing Virtual Machine (SDVM) is an adaptive, self con- 
figuring and self distributing virtual machine for clusters of heteroge- 
neous, dynamic computer resources. In this paper the concept and fea- 
tures of the implemented prototype is presented. 



1 Introduction 

State of the art systems of computers are mostly used to run sequential programs, 
though client-server-systems or parallelizing compilers are increasingly popular. 

So far, many parallel programs run on dedicated parallel computing ma- 
chines. Unfortunately, these multi-processor systems cannot be adapted to all 
needs of problems or algorithms to turn the inherent or explicit parallelism into 
efficiency. Therefore large clusters of PCs become important for parallel appli- 
cations. These clusters show a huge dynamic and heterogenity with a structure 
that is usually unknown at the compile time of the application. Furthermore, the 
CPUs of the cluster have immense changing loads, nodes are more or less specific 
and the network is spontaneous with vanishing and appearing resources. Besides, 
the reuse of existing hardware would be more economic and cost-efficient. 

Thus, mechanisms have to be developed to perform the (efficient) distribu- 
tion of code and data. They should be executable on arbitrary machines - not 
considering different processors and/or operating systems. 

Computers spending their machine time partly to the parallel calculations 
need applications running silently in the background without user interactions 
- like a UNIX-daemon. These computers should not be burdened too much by 
the background process to keep the foreground processes running smoothly. Full 
machine time must be available on demand for foreground processes, so the 
background process must support a shutdown at any time - or at least a nice 
feature to calm down its machine time consumption. Similarly the background 

* Parts of this work have been supported by the Deutsche Forschungsgemeinschaft 
(DFG). 
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process should be started by the user expecting idle times for a while. So it 
has to reintegrate into the cluster without the need for a restart of the parallel 
application. The cluster would then be self configuring regarding its composition. 

Some batch processing systems, as e. g. Condor [1], feature cycle harvesting 
- a method to identify idle times of the participating sites. The systems base 
on a client-server structure, in which the server decides about the distribution 
and execution of the scheduled jobs centrally. As all sites have to report to the 
server, the communication channel tends to be the bottleneck of the system 
especially in a loosely coupled cluster. The SDVM works without a client-server 
structure on a peer-to-peer basis, so neighborhood relations are automatically 
strengthened. 

Due to the size of large computer clusters the probability of hardware failures 
needs to be considered. Large clusters are hardly usable unless a concept for self 
healing of the cluster is available, making the parallel execution immune to 
hardware failures. 

To address these problems, the Self Distributing Virtual Machine (SDVM) 
has been developed and prototypically implemented in C++ by the authors. 



1.1 Goals 

The SDVM has been developed to introduce the following features into standard 
computer clusters. 

— self configuration: The system should adapt itself to different environments. 
Signing in and signing off at runtime should be available for all machines in 
the cluster. 

— adaptivity: The cluster should cope with heterogeneous computer systems, 
so computers with different speeds and even different operating systems can 
participate in an SDVM controlled cluster. 

— self distribution: Intelligent distributed scheduling and migration schemata 
should provide that data and instruction code is distributed among the clus- 
ter engines automatically at a minimum cost of machine time. 

— self healing: Crashes should be detected, handled and corrected by the SDVM 
automatically. 

— automatic parallelization: Hidden parallelism in programs should be detected 
and exploited. 

— self protection and protection of others: For a cluster working within an open 
network it is important to consider safety aspects. The authenticity of users 
and data has to be ensured, and sabotage, spying and corrumption has to 
be prevented. 

— accounting: Mechanisms and cost functions for accounting of provided and 
used computing time should be offered. 

The first three of these goals are addressed in this first paper about the 
SDVM. 
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1.2 Possible Fields of Application 

The SDVM can in the first instance be seen as a cheap opportunity to shorten 
the runtime of a program by parallelization. Due to self configuration (here: 
configuration of the computing cluster) it can be decided at run time to enlarge 
the cluster to get more performance or to downsize the cluster if participants 
are needed elsewhere. For example the use of a company’s workstation cluster 
at lunch time or at night is imaginable. 

Applied to the internet, the SDVM can be used to solve complex problems by 
collaboration of many computers, such as Seti@Home [2], In this way computers 
which are currently at the night hemisphere of the earth can join into an SDVM 
cluster and sign off in the morning. 

In contrast to the internet solution a monolithic version can be implemented. 
In this case the SDVM can be seen as a concept for a self distributing scheduling 
of calculations on a given multiprocessor system. 

1.3 VMs 

Generally a virtual machine is a software layer, which emulates the functionality 
of a certain machine or processor on another machine. Usually the user does 
not recognize the emulation. Virtual machines are used for example to capsulate 
program execution because of security and supervision reasons, so that the ap- 
plication gets only access to virtual resources, and the access is not forwarded 
to the real resources, unless the access is checked and granted. 

The meanwhile widely used Parallel Virtual Machine (PVM) [3] is essentially 
a collection of procedures and functions to simplify the usage of heterogeneous 
computer clusters. On each participating computer a daemon (pvmd) has to be 
started, which performs communication with other pvmd’s on different cluster 
computers via special functions. To run an PVM a “host pool” has to be config- 
ured before run time, with a constant total of machines and a fixed communica- 
tion infrastructure. The SDVM whereas allows signing in and out of computers 
at run time, without the need to know before run time which computers will 
participate. 



2 Concepts 

2.1 Microframes and Microthreads 

The SDVM is based on the concept of the Cache Only Memory Architectures 
(COMAs) [4], In a COMA architecture a processor which needs specific data, 
looks for it in its local memory. The local memory is active and checks whether 
the data is locally available. In case of a cache hit it returns the data to the 
processor. In case of a cache miss it autonomously connects to another computer 
in the COMA cluster, and asks for the data. The answer of this query will be 
written into the local memory and propagated to the processor. Thus the data 
access on COMA clusters is done transparently for the application. 
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Fig. 1 . Microframe and Microthread 



The SDVM extends this concept such, that beside the transparent data mi- 
gration also program instructions can transparently migrate. For the develop- 
ment of the SDVM the SDAARC architecture [5,6,7] is the origin. Program in- 
structions are represented by microthreads. A microthread contains a (for each 
computer architecture compiled) code fragment, but it lacks its start arguments 
for execution. 

The SDVM generally distinguishes two types of data: global data and mi- 
croframes. Global data in terms of COMA are spread over the sites participating 
in the SDVM cluster. Microframes as a special case of global data are containers 
for the arguments which a microthread needs to execute. They contain the code 
fragment, a pointer to the owning microthread, and addresses to microframes, 
which need the results of the microthread for their own execution. As soon as a 
microframe has all its arguments, it becomes executable and the corresponding 
microthread can be executed with the arguments of the microframe as parame- 
ters - thus the execution is triggered using dataflow synchronisation. Several mi- 
croframes can point to the same microthread (n-to- 1-relation). Both microframe 
and microthread have an unambiguous identifier each. Figure 1 shows a mi- 
croframe and its corresponding microthread. 

As we don’t have a compiler to split up an application to be run on the 
SDVM automatically into microthreads yet, currently the user has to do this 
by hand. However, this is not like typical parallel programming, since the user 
doesn’t have to worry about the binding of the microthreads to specific sites - 
this is done automatically by the SDVM at runtime. 

2.2 Execution Layer 

The SDVM is structured in two layers: an execution and a communication layer. 
Both layers consist of several modules (“managers”), which communicate over 
method calls. 

The main part of the execution layer (fig. 2) is the scheduling manager. It 
manages the executable microframes and tells the code manager to provide the 
corresponding microthreads. 

The code manager loads the microthread instructions from hard disk or ob- 
tains it from code managers of other sites. Then it calculates the correct start 
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Fig. 2. The execution layer of the SDVM 



address of the microthread code and returns it to the scheduling manager. If 
the microthread doesn’t exist in a compatible binary format, it is created by 
retrieving the C sourcecode of the microthread and compiling it with the locally 
installed C compiler. 

The processing manager supervises the actual execution by getting the mi- 
croframe and corresponding microthread from the scheduling manager, if it has 
free resources. It reads the start parameters for the microthread out of the mi- 
croframe and starts the execution of the microthread at the start address of the 
microthread provided by the scheduling manager. During the execution of the 
microthread the processing manager can access the global data memory by read- 
ing and writing. The results of the execution generate the arguments for further 
not yet executable microframes. The processing manager passes the results to 
the attraction memory (which behaves like the attraction memory in COMA 
[4]) together with the addresses of the microframes which are waiting for these 
results. 

The attraction memory writes the results to the destination addresses and 
checks if further microframes got all their arguments now and thus have become 
executable. In this case the executable microframes are sent to the scheduling 
manager. 



2.3 Communication Layer 

The message manager is the main part of the communication layer (see fig. 3). 
It is the contact point for all managers which need to communicate with other 
SDVM sites. A manager creates a message in a standardized format (SDMessage) 
and hands it over to the message manager. An SDMessage includes beside the 
destination site address and the destination manager id also the site address and 
the manager id of the originator. Also it contains an unambiguous identifier and 
the referenced data itself. 

The SDVM provides two communication modes: synchronous and asyn- 
chronous. If a message is sent synchronously, the originator waits and freezes 
until it receives an answer. The message manager sends the message and scans 
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Fig. 3. Within a site all managers communicate via the message manager with other 
managers on other sites. 



all incoming messages if there is the answer message for the synchronous com- 
munication. This answer is returned to the waiting manager directly as return 
value. Synchronous messages are especially intended for fast requests, so they 
have a preferred delivery, to freeze the originator as shortly as possible. 

An asynchronous message does not need an answer message. Asynchronous 
messages are just delivered to the message-in interface of a manager without 
freezing the originator. The message manager provides message bundeling for 
messages to the same target site to save communication overhead. For this fea- 
ture the message manager collects messages within a certain time window and 
then communicates the compressed message bundle. Synchronous messages are 
considered to be urgent, so they will be sent immediately. 

To send a message, the message manager needs to know the physical (IP) 
address of the site. As all sites have a uniform logical address (site id) obtained 
from the sign-in procedure, lookups are inevitable. This is done by the cluster 
manager which provides a list with logical and physical addresses of known sites. 

The message dispatching is performed by the network manager, which sends 
the message over the network to the destination site. After reception the message 
is processed by the remote network manager and forwarded to its local message 
manager for further distribution to the target manager. 

The structure of a site is shown in figure 3. 
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Fig. 4. Communication between sites 




site a site b 



□ 

□ 



global data 
microframes 



Fig. 5. Homesite directories contain pointers to locally created data objects (mi- 
croframes or global data) and all copies, if there are some. If an object migrates, 
its homesite updates the corresponding entry. 



2.4 Communication between Two Sites 

Components of distributed systems have to decide if data is stored locally or 
in a remote site. The SDVM is provided with homesite-directories to solve this 
problem. 

A homesite directory (see fig. 5) contains information regarding all locally 
created objects as there are microframes and global data. The global address of 
any of these objects includes the logical site address of the site it was created 
on. This site is called the homesite of the data object. When one of these objects 
is copied or migrated, the new location of the object or copies of it is written 
to the homesite directory. Therefore the object can be located at any time by 
querying its homesite. The sequence of an access to any object o is as follows, 
regarding site s: 
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1. On site s an access to object o is necessary. 

2. The attraction memory on site s starts a search in the local memory. If o is 
found, the access can be completed locally. 

3. If o is not found on site s, the attraction memory reads the homesite address 
from o (which is coded into the global address of o) , say h. 

4. Site s sends a message to site h containing a request for object o. 

5. Now site h searches in its local memory. If o is found, it is sent to site s and 
the address of s written to the homesite directory. 

6. If o was not found on site h either, the attraction memory scans its homesite 
directory to determine on which site o currently resides, say r. 

7. Site h informs site s that the object o resides on site r. 

8. Site s sends a message to site r containing a request for object o. 

9. Site r scans its local memory, locates o and sends it to site s. 

10. Finally site s informs the homesite h that object o now resides on Site s, 
and h updates the corresponding entry in its homesite directory. 

It may happen that object o is migrated from site r to another site while site 
s gets the information that o resides on site r. In this case site r would return 
an error, too, and site s would have to query the homesite h again, which in the 
meantime has updated its homesite directory. 

If object o has copies, the homesite has to be informed whenever there is a 
write access to o. The homesite then forwards the new value to all other sites 
storing a copy of o to keep the copies consistent. 

To save communications cost, the SDVM offers not to pull the object o to 
the writing site but to send the data to be written to the site where o resides. In 
this case o stays just there. This may make sense in cases when one site writes 
many times to object o and another site writes only a few times. 

2.5 Add Sites to or Remove Sites from the Cluster at Run Time 

To join a running SDVM cluster, a SDVM daemon has to be started on a machine 
and the physical (IP-) address of an active member of the cluster has to be given 
by the user (or a config file etc.). The SDVM daemon builds a new site which 
signs on to the given cluster-member. From there it gets its own logical site 
address. Then the new site is ready and asks for work. When the first executable 
microframe with the corresponding microthread is transferred to the new site, 
the new site is a member of the cluster. In this way the SDVM-cluster can be 
expanded arbitrarily at run time. 

If a machine is needed for another task, the local site s can vanish from the 
SDVM-cluster at any time. To do that, all locally stored microframes and data 
must be transferred to another member of the cluster. Moreover, another site t 
has to be declared the new homesite of this data. Site t receives the homesite 
directory of site s, includes it into its own homesite directory and from now on 
site t has more than one logical site address - its own and the newly received. 
Finally, all other sites are informed that the physical site address of site s has 
changed: The cluster managers receive the new physical site address of the new 
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homesite, so all queries and communications to site s are redirected to site t 
automatically. 

If a machine suddenly fails, more precisely if appears unreachable without 
having performed a regular sign off, a crash situation has occured. For this case, 
the SDVM prototype contains checkpointing, which saves all microframes and 
global data to a site in regular intervals. If a crash occurs, first of all the program 
has to be stopped by informing all involved sites. These delete all corresponding 
microframes and global data. Then the site with the most recent checkpoint is 
determined, and this site copies all saved data concerning this checkpoint to its 
global memory. Then it restarts execution of the program normally, and all other 
sites which are idle ask for work automatically, so that after a short period of 
time the program is distributed again. 



3 First Results 

The SDVM needs a lot of calculations and communications to distribute code and 
data. Therefore a question is whether the additional overhead is small enough 
to maintain the concept. 

To answer this question, an example SDVM-program was developed that 
is easily parallelizable. The example program does an integration using the 
Romberg numerical algorithm [8] . This algorithm partitions the area to be mea- 
sured into several portions of equal width. Those can be measured independently 
and the results added eventually. 

First of all, it shall be demonstrated how much overhead is generated by using 
the SDVM. To show this, run times on a stand-alone SDVM site are compared 
with the run times of a corresponding sequential program. This overhead appears 
to be about 3%. 



width 


accuracy 


SDVM (1 site) 


sequential program 


100 


20 


129s 


125s 


150 


20 


200s 


196s 



In the next step, it has to be shown that the speedup is in expected regions. 
On a cluster of identical machines (Pentium IV, 1700 MHz), a value for the 
speedup is: 



width 


accuracy 


site A 


site A+B (Speedup) 


site A+B+C (Speedup) 


100 


20 


119s 


63s (1,89) 


43s (2,77) 


150 


20 


179s 


96s (1,86) 


69s (2,59) 



Finally, the program was executed on three different machines respectively, 
and the execution times listed. 



width 


accuracy 


site D stand-alone 


site E stand-alone 


site F stand-alone 


100 


20 


107s 


129s 


281s 


150 


20 


163s 


200s 


426s 
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The execution times of different numbers and combinations of machines used 
to calculate are opposed to the previous results. The machine on which the 
program was started first is listed first, the others were added to the cluster at 
run time and so helped the first machine. 



width 


accuracy 


site D+E 


site D+E+F 


site F+D+E 


100 


20 




56s 


62s 


150 


20 


101s 


80s 





Obviously, the execution of the program started on the slowest machine (F) 
can be accelerated very much, because the helpers are faster than F. 

4 Conclusion and Future Prospects 

The SDVM is a platform to pool any computers in a dynamic cluster and run 
different microthreads, which are fragmented programs. The distribution of the 
microthreads is done automatically and adapts to given constraints. For working 
in the background, the SDVM is developed as a daemon. 

In a first implementation, the goals of self-distribution (microframes migrate 
to idle computers on their own), self-configuration (computers can sign on and 
sign off at the cluster at run time) and adaptivity (different hardware for the 
participating machines is supported) are reached. Furthermore, the concurrent 
execution of several SDVM-programs is supported. 

Presently, the integration of a security concept (self-protection) and crash- 
management (self-healing) is in preparation. 

Future versions could provide self-distributing in-/output, e.g. for users, who 
access the same cluster from different terminals, or for a chat program. Also, it is 
possible to integrate other transfer protocols - or a protocol specifically designed 
for the SDVM - beside or instead of the currently used TCP/IP, to address 
routing characteristics of the cluster or even optimize the routing dynamically. 

Furthermore, a compiler may be created which partitions any sequential pro- 
gram automatically and thus completes the path from a sequential program to 
a self-distributing parallely executed program. 



References 

1. Litzkow, M., Livny, M., Mutka, M.: Condor - a hunter of idle workstations. In: 
Proceedings of the 8th International Conference of Distributed Computing Systems. 
(1988) 

2. Sullivan III, W.T., Werthimer, D., Bowyer, S., Cobb, J., Gedye, D., Anderson, D.: 
A new major SETI project based on Project Serendip data and 100,000 personal 
computers. In Cosmovici, C., Bowyer, S., Werthimer, D., eds.: Astronomical and 
Biochemical Origins and the Search for Life in the Universe, Proceedings of the 
Fifth International Conference on Bioastronomy. Editrice Compositori, Bologna, 
Italy (1997) http : / /setiathome . ssl .berkeley . edu7woody_paper.html. 



The SDVM: A Self Distributing Virtual Machine for Computer Clusters 



19 



3. Sunderam, V.S.: PVM: A framework for parallel distributed computing. In: Con- 
currency: Practice and Experience. (1990) 315-339 

4. Hagersten, E., Landin, A., Haridi, S.: DDM — A Cache-Only Memory Architecture. 
IEEE Computer 25 (1992) 44-54 

5. Moore, R., Klauer, B., Waldschmidt, K.: The SDAARC architecture. In: Proceed- 
ings of the Ninth Euromicro Workshop on Parallel and Distributed Processing (PDP 
2001), Mantova, Italy, IEEE Computer Society Press (2001) 429-435 

http : //www. ti . inf ormatik.uni-frankfurt . de/Paper s/Adar c/mant ova. pdf . 

6. Moore, R., Klauer, B., Waldschmidt, K.: Tailoring a self-distributing architecture 
to a cluster computer environment. In: 8th Euromicro Workshop on Parallel and 
Distributed Processing (EURO-PDP 2000), Rhodes, Greece, IEEE Computer Soci- 
ety Press (2000) 

http : //www. ti . inf ormatik.uni-frankfurt . de/Paper s/Adar c/rhodos .pdf. 

7. Eschmann, F., Klauer, B., Moore, R., Waldschmidt, K.: SDAARC: An Extended 
Cache-Only Memory Architecture. IEEE micro 22 (2002) 62-70 

8. Dahlquist, G., Bjorck, A.: Numerical Methods. Prentice Hall, Englewood Cliffs, NJ 
(1974) 



Heterogenous Data Fusion via a Probabilistic 
Latent- Variable Model 



Kai Yu and Volker Tresp 

Information and Communications, 
Siemens Corporate Technology, 
Otto-Hahn-Ring, 6 
81730 Munich, Germany 



Abstract. In a pervasive computing environment, one is facing the 
problem of handling heterogeneous data from different sources, trans- 
mitted over heterogeneous channels and presented on heterogeneous user 
interfaces. This calls for adaptive data representations keeping as much 
relevant information as possible while keeping the representation as small 
as possible. Typically, the gathered data can be high-dimensional vec- 
tors with different types of attributes, e.g. continuous, binary and cat- 
egorical data. In this paper we present - as a first step - a probabilis- 
tic latent-variable model, which is capable of fusing high-dimensional 
heterogenous data into a unified low-dimensional continuous space, and 
thus brings great benefits for multivariate data analysis, visualization 
and dimensionality reduction. We adopt a variational approximation to 
the likelihood of observed data and describe an EM algorithm to fit the 
model. The advantages of the proposed model are illustrated on toy data 
and used on real-world painting image data for both visualization and 
recommendation. 



1 Introduction 

Among others, pervasive computing will be characterized by the processing of 
heterogeneous and high-dimensional data. For example, results provided by In- 
ternet search engines may contain text, pictures, hyperlinks, categorial and bi- 
nary data. The demand for clearly structured information presented to end users, 
but also the limitations of telecommunication networks as well as user interfaces 
calls for a lower-dimensional representation providing the most relevant informa- 
tion. Promising candidates for this task of dimensionality reduction are latent 
variable models. 

Latent variable analysis is a family of data modelling approaches that fac- 
torizes high-dimensional observations with a reduced set of latent variables. The 
latent variables offer explanations of the dependencies between observed vari- 
ables. An example is the probabilistic variant of the widely used principal com- 
ponent analysis, PPCA, where observations are explained by a linear projection 
of a set of Gaussian hidden variables, plus additive observation noise [10]. Stan- 
dard PCA is widely used for data reduction, pattern recognition and exploratory 
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data analysis. Recent studies on PC A reveals its connections to statistical factor 
analysis (FA) [7]. 

While existing PCA or FA approaches rely on continuous-valued observa- 
tions, data analysis on mixed types of data (discrete and continous observations) 
is often desirable: 

— In solving typical data mining problems, one is always faced with mixed data. 
For example, a hospital patient’s record typically includes fields like age (dis- 
crete real- valued) , gender (binary), various examination results (real- valued 
or categorical), binary indicator variables for the presence of symptoms or 
even textual descriptions. A unified means to explore the dependencies of 
these data are needed. 

— If applied to dimensionality reduction for pattern recognition, PCA is purely 
unsupervised. Thus, the resulting projection can be not indicative of the tar- 
geted pattern distribution. A generalized PCA which allows class member- 
ship as additional attributes (binary or categorical) may obviously provide 
a better solution. 

— For heterogeneous data, it is often difficult to derive a small set of com- 
mon features describing the total data. For example, in a web-based image 
retrieval system, each image can be characterized by its visual features, ac- 
companying words, categories, and user visit records. 

For these reasons, we will present a probabilistic latent variable model to fit 
observations with both continuous and binary attributes in this paper. Since 
categorical attributes can always be encoded by sets of binary attributes 1 (e.g. 1- 
of-c coding scheme), this model can be applied to a wide range of situations. We 
call this model generalized probabilistic PCA, GPPCA. 

In the next section we describe the latent variable model and derive an effi- 
cient variational expectation-maximization (EM) formalism to learn the model 
from data. In Sec. 3 we discuss properties of the model and connections to pre- 
vious work. In Sec. 5 we present empirical results based on toy data and image 
data, with focus on both data visualization and information filtering. 



2 A Generalized Probabilistic PCA Model 

The goal of a latent variable model is to find a representation for the distribution 
p(t) of observed data in an M-dimensional space t = (t±, . . . , <m) hr terms of 
a number of L latent variables x = (x±, In our setting of interest, we 

consider a total of M continuous and binary attributes. We use m € 77. to indicate 
that the variable t rn is continuous- valued, and m £ B for binary variables (i.e. {0, 
1}). The generative model is: 



1 To be precise, an additional constraint is required here, which we drop for simplicity. 



